Method and apparatus for video encoding and decoding using pattern-based block filtering

ABSTRACT

Methods ( 1100, 1300 ) and apparatuses ( 600, 1200 ) for video coding and decoding are provided. The method of video encoding includes accessing ( 1110 ) a reconstructed block corresponding to a block in a picture of a video, determining ( 1120 ) at least one filter pattern based on a property of the block and filtering ( 1130 ) the reconstructed block according to the at least one filter pattern. The method of video decoding includes accessing ( 1310 ) a reconstructed block corresponding to a block in a picture of an encoded video, determining ( 1320 ) at least one filter pattern based on a property of the block and filtering ( 1330 ) the reconstructed block according to the at least one filter pattern. A bitstream formatted to include encoded data, a computer-readable storage medium and a computer program product are also described.

This application claims the benefit, under 35 U.S.C. § 365 ofInternational Application PCT/US2018/049733, filed Sep. 6, 2018, whichwas published in accordance with PCT Article 21(2) on Mar. 14, 2019, inEnglish, and which claims the benefit of European Patent Application No.17306162.3, filed Sep. 8, 2017.

TECHNICAL FIELD

The present embodiments generally relate to video encoding and decoding,particularly to pattern-based block filtering of reconstructed blocks.

BACKGROUND

Any background information described herein is intended to introduce thereader to various aspects of art, which may be related to the presentembodiments that are described below. This discussion is believed to behelpful in providing the reader with background information tofacilitate a better understanding of the various aspects of the presentdisclosure. Accordingly, it should be understood that these statementsare to be read in this light.

To achieve high compression efficiency, image and video coding schemesusually employ prediction and transform to leverage spatial and temporalredundancy in the video content. Generally, intra or inter prediction isused to exploit the intra or inter frame correlation, then thedifferences between the original image and the predicted image, oftendenoted as prediction errors or prediction residuals, are transformed,quantized and entropy coded. To reconstruct the video, the compresseddata is decoded by inverse processes corresponding to the prediction,transform, quantization and entropy coding.

Post-filtering may be performed after reconstruction of the picture orimage to smooth it and reduce noise and coding artifacts whilepreserving the edges. More recently, block filtering (or in-loop-blockfiltering) may be performed for each reconstructed block of the picture.FIG. 1 illustrates simplified flowcharts of methods of block filtering100 versus post-filtering 150 of reconstructed blocks of a picture (orslice of a picture) according to the prior art. In post-filtering 150,the filtering 180 is performed on all (or a group of, e.g., a slice)blocks after all (or a group of) the blocks in the picture are accessed160, decoded and reconstructed 170. In block filtering 100, each blockis filtered 130 after being accessed 110, decoded and reconstructed 120.

The use of block filtering 100 results in some advantages. First, thefiltered blocks may be used for predicting the next and/or neighboringblocks (Intra prediction), resulting in improved prediction. Second, thecoding modes are chosen using rate-distortion optimization (RDO) on thefiltered reconstructed blocks, after elimination of artifacts, resultingin improved selection.

One example of block filtering is a bilateral filter (BLF), which is anon-linear, edge-preserving, and noise-reducing smoothing filter forimages. It replaces the intensity of a pixel with a weighted average ofintensity values from nearby pixels. The weights may be based on aGaussian distribution. The current Joint Video Exploration Team (JVET)Joint Exploration test Model (JEM) reference software (“AlgorithmDescription of Joint Exploration Test Model 5 (JEM 5)”, by Jianle Chenet al.) utilizes bilateral filtering per block of the picture, morespecifically, per coding unit (CU) (Document JVET-F0034, by J. Ström etal., entitled “EE2-JVET related: Division-free bilateral filter,” JVETof ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 6th Meeting: Hobart,AU, 31 Mar.-7 Apr. 2017).

FIG. 2 illustrates the filtering cross pattern per block 200 of thepicture in accordance with the JVET-F0034 document of the prior art. Foreach pixel or sample inside the block (e.g., current pixel C 210), theBLF performs the filtering based on a specific neighboring shapeconsisting of four nearest neighbors (220-250) of the pixel (210) tofilter on a cross (or plus, “+”) pattern. The BLF computes a weightedsum of the four cross-patterned neighboring samples or pixels and thecenter or current pixel.

SUMMARY

According to an aspect of the present disclosure, a method of videoencoding is provided including accessing a reconstructed blockcorresponding to a block in a picture of a video, determining at leastone filter pattern based on a property of the block and filtering thereconstructed block according to the at least one filter pattern.

According to an aspect of the present disclosure, an apparatus for videoencoding is provided, the apparatus including means for accessing areconstructed block corresponding to a block in a picture of a video,means for determining at least one filter pattern based on a property ofthe block and means for filtering the reconstructed block according tothe at least one filter pattern.

According to an aspect of the present disclosure, an apparatus for videoencoding is provided, the apparatus including a processor, and at leastone memory coupled to the processor, the processor being configured toaccess a reconstructed block corresponding to a block in a picture of avideo, determine at least one filter pattern based on a property of theblock and filter the reconstructed block according to the at least onefilter pattern.

According to an aspect of the present disclosure, a bitstream formattedto include encoded data representative of a block of a picture, theencoded data encoded by accessing a reconstructed block corresponding toa block in a picture of a video, determining at least one filter patternbased on a property of the block and filtering the reconstructed blockaccording to the at least one filter pattern.

According to an aspect of the present disclosure, a signal including abitstream formatted to include encoded data representative of a block ofa picture, the encoded data encoded by accessing a reconstructed blockcorresponding to a block in a picture of a video, determining at leastone filter pattern based on a property of the block and filtering thereconstructed block according to the at least one filter pattern.

According to an aspect of the present disclosure, a method of videodecoding is provided including accessing a reconstructed blockcorresponding to a block in a picture of an encoded video, determiningat least one filter pattern based on a property of the block andfiltering the reconstructed block according to the at least one filterpattern.

According to an aspect of the present disclosure, an apparatus for videodecoding is provided, the apparatus including means for accessing areconstructed block corresponding to a block in a picture of an encodedvideo, means for determining at least one filter pattern based on aproperty of the block and means for filtering the reconstructed blockaccording to the at least one filter pattern.

According to an aspect of the present disclosure, an apparatus for videodecoding is provided, the apparatus including a processor, and at leastone memory coupled to the processor, the processor being configured toaccess a reconstructed block corresponding to a block in a picture of anencoded video, determine at least one filter pattern based on a propertyof the block and filter the reconstructed block according to the atleast one filter pattern.

According to an aspect of the present disclosure, a computer programproduct is provided including program code instructions for accessing areconstructed block corresponding to a block in a picture of a video,determining at least one filter pattern based on a property of the blockand filtering the reconstructed block according to the at least onefilter pattern.

According to an aspect of the present disclosure, a computer programproduct is provided including program code instructions for accessing areconstructed block corresponding to a block in a picture of an encodedvideo, determining at least one filter pattern based on a property ofthe block and filtering the reconstructed block according to the atleast one filter pattern.

According to an aspect of the present disclosure, a computer-readablestorage medium carrying a software program is provided including programcode instructions for accessing a reconstructed block corresponding to ablock in a picture of a video, determining at least one filter patternbased on a property of the block and filtering the reconstructed blockaccording to the at least one filter pattern.

According to an aspect of the present disclosure, a computer-readablestorage medium carrying a software program is provided including programcode instructions for accessing a reconstructed block corresponding to ablock in a picture of an encoded video, determining at least one filterpattern based on a property of the block and filtering the reconstructedblock according to the at least one filter pattern.

The above presents a simplified summary of the subject matter in orderto provide a basic understanding of some aspects of subject matterembodiments. This summary is not an extensive overview of the subjectmatter. It is not intended to identify key/critical elements of theembodiments or to delineate the scope of the subject matter. Its solepurpose is to present some concepts of the subject matter in asimplified form as a prelude to the more detailed description that ispresented later.

Additional features and advantages of the present disclosure will bemade apparent from the following detailed description of illustrativeembodiments which proceeds with reference to the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure may be better understood in accordance with thefollowing exemplary figures briefly described below:

FIG. 1 illustrates simplified flowcharts of methods of block filtering(on the left) versus post-filtering (on the right) of reconstructedblocks of a picture in accordance with the prior art;

FIG. 2 illustrates a cross pattern of neighboring samples used in abilateral filter in accordance with the prior art;

FIG. 3 illustrates a CTU split into CUs in accordance with the HEVCstandard;

FIG. 4 illustrates the splitting of a CTU into CUs, PUs and TUs inaccordance with the HEVC standard;

FIG. 5 illustrates a CTU in accordance with the QTBT tool;

FIG. 6 illustrates a simplified block diagram of an exemplary videoencoder in accordance with an embodiment of the present disclosure;

FIG. 7 illustrates a diagonal pattern of neighboring samples used in afilter in accordance with an embodiment of the present disclosure;

FIG. 8 illustrates a picture including blocks suited for a diagonalpattern of neighboring samples in accordance with an embodiment of thepresent disclosure;

FIG. 9A illustrates a square pattern of neighboring samples used in afilter in accordance with an embodiment of the present disclosure;

FIG. 9B illustrates a vertical-cross, a horizontal-cross, adiagonal-left and a diagonal-right pattern used in a filter inaccordance with an embodiment of the present disclosure;

FIG. 10 illustrates some exemplary partial patterns of neighboringsamples used in a filter in accordance with an embodiment of the presentdisclosure;

FIG. 11 illustrates a flowchart of an exemplary method of encoding inaccordance with an embodiment of the present disclosure;

FIG. 12 illustrates a simplified block diagram of an exemplary videodecoder in accordance with an embodiment of the present disclosure;

FIG. 13 illustrates a flowchart of an exemplary method of decoding inaccordance with an embodiment of the present disclosure; and

FIG. 14 illustrates a block diagram of a computing environment withinwhich aspects of the present disclosure can be implemented and executed.

DETAILED DISCUSSION OF THE EMBODIMENTS

It should be understood that the elements shown in the figures may beimplemented in various forms of hardware, software or combinationsthereof. Preferably, these elements are implemented in a combination ofhardware and software on one or more appropriately programmedgeneral-purpose devices, which may include a processor, memory andinput/output interfaces. Herein, the phrase “coupled” is defined to meandirectly connected to or indirectly connected with through one or moreintermediate components. Such intermediate components may include bothhardware and software based components.

The present description illustrates the principles of the presentdisclosure. It will thus be appreciated that those skilled in the artwill be able to devise various arrangements that, although notexplicitly described or shown herein, embody the principles of thedisclosure and are included within its scope.

All examples and conditional language recited herein are intended foreducational purposes to aid the reader in understanding the principlesof the disclosure and the concepts contributed by the inventor tofurthering the art, and are to be construed as being without limitationto such specifically recited examples and conditions.

Moreover, all statements herein reciting principles, aspects, andembodiments of the disclosure, as well as specific examples thereof, areintended to encompass both structural and functional equivalentsthereof. Additionally, it is intended that such equivalents include bothcurrently known equivalents as well as equivalents developed in thefuture, i.e., any elements developed that perform the same function,regardless of structure.

Thus, for example, it will be appreciated by those skilled in the artthat the block diagrams presented herein represent conceptual views ofillustrative circuitry embodying the principles of the disclosure.Similarly, it will be appreciated that any flow charts, flow diagrams,state transition diagrams, pseudocode, and the like represent variousprocesses which may be substantially represented in computer readablemedia and so executed by a computer or processor, whether or not suchcomputer or processor is explicitly shown.

The functions of the various elements shown in the figures may beprovided through the use of dedicated hardware as well as hardwarecapable of executing software in association with appropriate software.When provided by a processor, the functions may be provided by a singlededicated processor, by a single shared processor, or by a plurality ofindividual processors, some of which may be shared. Moreover, explicituse of the term “processor” or “controller” should not be construed torefer exclusively to hardware capable of executing software, and mayimplicitly include, without limitation, digital signal processor (DSP)hardware, read only memory (ROM) for storing software, random accessmemory (RAM), and nonvolatile storage.

Other hardware, conventional and/or custom, may also be included.Similarly, any switches shown in the figures are conceptual only. Theirfunction may be carried out through the operation of program logic,through dedicated logic, through the interaction of program control anddedicated logic, or even manually, the particular technique beingselectable by the implementer as more specifically understood from thecontext.

In the claims hereof, any element expressed as a means for performing aspecified function is intended to encompass any way of performing thatfunction including, for example, a) a combination of circuit elementsthat performs that function or b) software in any form, including,therefore, firmware, microcode or the like, combined with appropriatecircuitry for executing that software to perform the function. Thedisclosure as defined by such claims resides in the fact that thefunctionalities provided by the various recited means are combined andbrought together in the manner which the claims call for. It is thusregarded that any means that can provide those functionalities areequivalent to those shown herein.

It is to be understood that the figures and descriptions have beensimplified to illustrate elements that are relevant for a clearunderstanding of the present disclosure, while eliminating, for purposesof clarity, many other elements found in typical encoding and/ordecoding devices.

It will be understood that, although the terms first and second may beused herein to describe various elements, these elements should not belimited by these terms. These terms are only used to distinguish oneelement from another. Various methods are described above, and each ofthe methods comprises one or more steps or actions for achieving thedescribed method. Unless a specific order of steps or actions isrequired for proper operation of the method, the order and/or use ofspecific steps and/or actions may be modified or combined.

It is to be understood that a picture is an array of luma samples inmonochrome format, or an array of luma samples and two correspondingarrays of chroma samples in 4:2:0, 4:2:2, and 4:4:4 color format. InHEVC, a “block” addresses a specific area in a sample array (e.g., lumaY), and a “unit” includes the collocated block of all encoded colorcomponents (luma Y and possibly chroma Cb and chroma Cr), syntaxelements and prediction data that are associated with the block (e.g.,motion vectors). However, the term “block” is more generally used hereinto refer to a block (e.g. a coding block (CB), transform block (TB),coding group (CG), etc.) or a unit (e.g. a CU).

It is to be understood that a picture or block of pixels or transformcoefficients is a two-dimensional array or matrix. The horizontal or xdirection (or axis) represents a width and the vertical or y direction(or axis) represents a height. The indexes start at 0. The x directionrepresents columns and the y direction represents rows. The maximum xindex is the width−1. The maximum y index is the height−1.

In the following sections, the word “reconstructed” and “decoded” may beused interchangeably. Usually but not necessarily “reconstructed” isused on the encoder side while “decoded” is used on the decoder side.Also, the words “coded” and “encoded” may be used interchangeably.Moreover, the words “image”, “picture” and “frame” may be usedinterchangeably. Furthermore, the words “coding”, “source coding” and“compression” may be used interchangeably.

The present disclosure addresses some disadvantages present in the priorart. In the JVET-F0034 document of the prior art, the BLF computes aweighted sum of the four cross-patterned neighboring samples or pixelsand the center or current pixel. However, while having a smallcomplexity, a simple cross pattern may limit the number of samples usedfor filtering and, as a result, decrease the efficiency of the BLF. Thepresent disclosure is directed to filtering patterns of neighboringsamples used in block filtering (e.g., bilateral filtering) that takeinto consideration the different shapes and properties of the codingblocks. A pattern is a specific shape comprising a current sample to befiltered and neighboring samples, whose values are used as inputs forfiltering the current sample. It can also be considered as the filtersupport.

In the High Efficiency Video Coding (HEVC) standard (“ITU-T H.265Telecommunication standardization sector of ITU (10/2014), series H:audiovisual and multimedia systems, infrastructure of audiovisualservices—coding of moving video, High efficiency video coding,Recommendation ITU-T H.265”), a picture is partitioned into coding treeunits (CTU) of square shape with a configurable size typically 64×64,128×128, or 256×256. As illustrated in FIG. 3, a CTU 310 is the root ofa quad-tree partitioning into leaves called Coding Units (CU). For eachCU, a prediction mode is signaled which indicates whether the CU iscoded using intra or inter prediction. As illustrated in FIG. 4, aconsecutive set of CTUs (e.g., CTU 420) may be grouped into a slice 410.A CU (e.g., CU 430) may be partitioned into one or more Prediction Units(PU) and forms the root of a quad-tree (known as transform tree)partitioning into Transform Units (TUs). Asymmetric subdivision of theCU into PUs is also possible in inter prediction, that is if a CU has asize N×N, a PU may have a size N/4×N, 3N/4×N, N×N/4, N×3N/4. Each PU isassigned some prediction information, for instance motion information,spatial intra prediction, etc.

The Quad-Tree plus Binary-Tree (QTBT) coding tool (DocumentJVET-C1001_v3, entitled “Algorithm Description of Joint Exploration TestModel 3”, Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11, 3rdmeeting, 26th May-1 Jun. 2015, Geneva, CH) is a new video coding toolthat provides a more flexible CTU representation and increasedcompression efficiency compared to the CU/PU/TU arrangement of the HEVCstandard. As illustrated in FIG. 5, the Quad-Tree plus Binary-Tree(QTBT) coding tool defines a coding tree 510 where coding units can besplit both in a quad-tree and in a binary-tree fashion. An exemplarycoding tree representation of a Coding Tree Unit 520 is illustrated inFIG. 5, where solid lines indicate quad-tree partitioning and dottedlines indicate binary partitioning of a CU 530 within CTU 520, which isspatially embedded in the quad-tree leaves.

The splitting of a CTU into coding units is decided on the encoder side,e.g. through a rate distortion optimization procedure which consists indetermining the QTBT representation of the CTU with minimal ratedistortion cost. In the QTBT representation, a CU has either a square ora rectangular shape. The size of a coding unit is always a power of 2,and typically goes from 4 to 128. The QTBT decomposition of a CTUcomprises two stages: the CTU is first split into 4 CUs in a quad-treefashion, then each quad-tree leaf can be further divided into two CUs ina binary fashion or into 4 CUs in a quad-tree fashion, as illustrated inFIG. 3.

With the QTBT representation, a CU may not be further partitioned intoPUs or TUs. In other words, each CU is considered as a single predictionunit and a single transform unit and such a QTBT representation onlyallows for symmetric splitting of a CU. More recently, however, the EPapplication No. 16306308.4, filed on Oct. 5, 2016 and entitled “ENCODINGAND DECODING METHODS AND CORRESPONDING DEVICES” describes CUs with newrectangular shapes which result from a new Binary Splitting Mode calledasymmetric splitting mode.

The present disclosure is directed to techniques for video or imageencoding and decoding (also known as source coding and decoding) whereblocks of a plurality of shapes and splitting modes may be allowed inthe video coding, that is, the encoder may choose any of these shapesand splitting modes and signal them to the decoder. The rich set of CUtopologies result in coding structures that spatially match thestructures and discontinuities contained in the images of a bitstream.

Encoding

FIG. 6 illustrates a simplified block diagram of exemplary video encoder600 in accordance with an embodiment of the present disclosure. Theencoder 600 may be included in a transmitter or headend in acommunication system. To encode a video sequence with one or morepictures, a picture may be partitioned into CTUs of square shape with aconfigurable size. A consecutive set of CTUs may be grouped into aslice. A CTU is the root of a QTBT partitioning into CUs. In theexemplary encoder 600, a picture is encoded by the encoder modules asdescribed below. Each block is encoded using either an intra mode orinter mode. When a block is encoded in an intra mode, the encoder 600performs intra prediction (module 660). In an inter mode, motionestimation (module 675) and compensation (module 670) are performed. Theencoder decides (module 605) which one of the intra mode or inter modeto use for encoding the block, and indicates the intra/inter decision bya prediction mode flag. Residuals are calculated by subtracting (module610) a predicted sample block (also known as a predictor) from theoriginal image block.

As an example, blocks in intra mode are predicted from reconstructedneighboring samples. Inter prediction is performed by performing motionestimation (module 675) and motion-compensating (in module 670) areference block stored in a reference picture buffer 680.

The residuals are transformed (module 625) and quantized (module 630).The transform module 625 may transform the image from the pixel or timedomain to the transform or frequency domain. The transform may be maybe, e.g., a cosine transform, a sine transform, a wavelet transform,etc. Quantization may be performed according to, e.g., a rate distortioncriterion. The quantized transform coefficients, as well as motionvectors and other syntax elements, are entropy coded (module 645) tooutput a bitstream. The entropy coding may be, e.g., Context AdaptiveBinary Arithmetic Coding (CABAC), Context Adaptive Variable LengthCoding (CAVLC), Huffman, arithmetic, exp-Golomb, etc. The encoder mayalso skip the transform and apply quantization directly to thenon-transformed residual signal. The encoder may also bypass bothtransform and quantization, i.e., the residual is coded directly withoutthe application of the transform or quantization process. In direct PCMcoding, no prediction is applied and the block samples are directlycoded into the bitstream.

The encoder comprises a decoding loop and thus decodes an encoded blockto provide a reference for further predictions. The quantized transformcoefficients are de-quantized (module 640) and inverse transformed(module 650) to decode residuals. An image block is reconstructed bycombining (module 655) the decoded residuals and the predicted sampleblock. A block filter 685 may filter the reconstructed block (e.g.,using a bilateral filter) after combiner (also called reconstructionmodule) 665. An in-loop filter(s) (i.e., a filter within the predictionloop, module 665) may be applied to the block filtered reconstructedpicture, including, for example, to perform deblocking/Sample AdaptiveOffset (SAO) filtering to reduce coding artifacts. In in-loop filtering,the filtering process may be performed, e.g., after an entire slice orimage/picture/frame has been reconstructed, all-in-one, so that thefiltered samples can be used for Inter-prediction. Hence, post-filtering150 may be applied to in-filter(s) 665. Notice, however, that blockfiltering 100 may also be utilized in in-loop filter(s) 665. Thefiltered image is stored in the reference picture buffer 680.

The modules of video encoder 600 may be implemented in software andexecuted by a processor, or may be implemented by well-known circuits byone skilled in the art of compression. In particular, video encoder 600may be implemented as an integrated circuit (IC).

The modules of video encoder 600 are also present in other videoencoders (e.g., HEVC encoders), except for the differences described inthe present disclosure, particularly, differences in the block sizes andshapes, and differences in the presence of block filter module 685, aswill be described in greater detail in the following paragraphs andfigures.

It is to be understood that, in additional (not shown) embodimentsaccording to the present disclosure, block filter 685 may be placed inone of: inside the intra prediction module 660, inside in-loop filter(s)module 665, inside both the intra prediction module 660 and the in-loopfilter(s) module 665, or inside the combiner module 655.

Regarding block filter module 685, in order to improve the performanceof the video encoders, it has been proposed to filter the reconstructedsignal with a filter that can reduce both the original signal noise (notpredictable and not desirable to encode in general) and thereconstructed signal coding artifacts, while preserving the edges. TheBilateral-Filter (BLF) allows fulfilling these requirements. BLF worksby basing the filter weights or taps not only on the distance toneighboring samples but also on their values. Each sample in thereconstructed picture is replaced by a weighted average of itself andits neighbors. The weights are calculated based on the distance from thecenter sample as well as the difference in sample values. Because thefilter is in the shape of a small plus sign as shown in FIG. 2, all ofthe distances may be 0 or 1. In one embodiment according to the presentdisclosure, other values may be chosen with a proper scale factor.

In the JVET group, it has been shown that BLF may allow the BjøntegaardDelta-Rate (BD-rate) gains about 0.41% in all intra (AI) and 0.48% inrandom access (RA) (Common Test Conditions) in Luminance, using the JEMreference encoder software.

However, one limitation of applying BLF on reconstructed blocks is itsrelative complexity. The BLF implies computing for each reconstructed(Luma) sample some weights associated with neighboring samples which arefunction of:

-   -   The distance of the neighboring samples (k,l) to the current        sample (i,j) to filter    -   The difference of the current sample value I(i,j) and the        neighboring sample values I(k, l)

A sample located at (i, j), will be filtered using its neighboringsample (k, l). The weight ω(i, j, k, l) is the weight assigned forsample (k, l) to filter the sample (i, j), and it is defined as:

$\begin{matrix}{{\omega\left( {i,j,\ k,\ l} \right)} = e^{({{- \frac{{({i - k})}^{2} + {({j - l})}^{2}}{2\sigma_{d}^{2}}} - \frac{{{{I{({i,j})}} - {I{({k,l})}}}}^{2}}{2\;\sigma_{r}^{2}}})}} & (1)\end{matrix}$

where σ_(d) is the spatial parameter, and σ_(r) is the range parameter.The properties (or strength) of the bilateral filter are controlled bythe two parameters, σ_(d) and σ_(r). In the JEM reference software,σ_(d) is set dependent on the transform unit size and prediction mode,and σ_(r) is set based on the quantization parameter (QP) used for thecurrent block.

The filtered samples I_(F)(i,j) are calculated as:I _(F)(i,j)=Σ_(k,l) I(k,l)*ω(i,j,k,l)/Σ_(k,l)ω(i,j,k,l)  (2)

The complexity due to accessing neighboring samples and in computing theweights may be alleviated by limiting the number of neighbors andtabulating the weights (using 1D-look up tables). In the WET-F0034document, four neighbors only (plus or cross pattern in FIG. 2) wereproposed so that the value of (i−k)²+(j−l)² is equal to 1 and the numberof tabulated values of ∥I(i, j)−I(k, l)∥² can be reduced to the samplerange values (1024 values for 10-bits).

Due to the fact that the filter only processes the current sample 210and its four neighbours, equation 2 may be written as:I _(F)(i,j)=(I _(C)ω_(C) +I _(L)ω_(L) +I _(R)ω_(R) +I _(A)ω_(A) +I_(B)ω_(B)/ω_(C)+ω_(L)+ω_(R)+ω_(A)+ω_(B)  (3)where I_(C) is the intensity of the center sample 210, and I_(L), I_(R),I_(A) and I_(B) are the intensities for the left 220, right 240,above250 and below 230 samples, respectively. Likewise, ω_(C) is theweight for the center sample 210, and ω_(L), ω_(R), ω_(A) and ω_(B) arethe corresponding weights for the neighbouring samples 220,240, 250 and230, respectively. The filter only uses samples within the block forfiltering—weights outside are set to 0.

The BLF allows for reducing the noise by averaging the current samplevalue 210 with neighboring sample values (220-250) when they are close.When the values of the neighbors are (very) different, the weights aresmall and their contribution is also small, so that the edges tend to bepreserved. In the WET-F0034 document, the process is appliedsystematically to all blocks, and to all pixels in the block, with afixed filter shape.

The use of the cross pattern (samples 220-250) reduces the complexitybut also limits the number of samples used for filtering, decreasing theefficiency of the BLF as a result. In particular, in the case ofdiagonal gradients, the neighbors whose values are close to the currentsample value are not used in the cross pattern.

According to the present disclosure, pattern diversity is introduced forperforming the block filtering 685 and for computing the filter weights.The pattern may be explicitly signaled in the stream, or it may beinferred or determined at the decoder depending on the characteristicsof reconstructed samples, using pixel classification for instance. Thepattern may be fixed for a current picture, for a current block, for allsamples in a block, or may change for each picture, each block, eachsub-block (e.g., 4×4 sub-block) and/or each sample in a block.

In the present disclosure, at least two steps are performed. A firststep performs the pattern selection. Then the block filter 685 (e.g.,BLF) is applied according to the selected pattern. The selection permitsimproving the performance of the overall video compression throughimprovement of the block filtering as a function of the signalcharacteristics.

FIG. 7 illustrates a diagonal pattern of neighboring samples used in afilter in accordance with an embodiment of the present disclosure. Thediagonal pattern has the shape of a symmetric “X”, which represents thecrossing of two block diagonals, a left leaning (or left) diagonal and aright leaning (or right) diagonal. In the diagonal pattern, the corners720-750 of the current sample 710 may be the closest in value to thecurrent sample 710 and are included in the weighted sum of the BLF.

The diagonal pattern re-uses most of the prior art block filteringoperations in FIG. 1 and equations 1-3 taking the difference in patterninto consideration. The diagonal pattern is better suited for blockscontaining local textures mostly oriented along diagonal directions,such as many portions of the picture 800 in FIG. 8. FIG. 8 is a picturefrom a reference sequence of Common Test Conditions used by the MovingPicture Experts Group (MPEG) body.

Both the cross and diagonal patterns use the same number of weightedsamples (four) so that the complexity is the same as using only onesingle pattern. Consequently, the existing codec design can be maximallyreused, with a small added complexity or implementational cost, wherethe added complexity is associated with the pattern selection process.

The selection may include selecting one of the cross pattern and thediagonal pattern. Other patterns may be considered, albeit with a highercomplexity. For example, a square pattern may be included in theselection process. FIG. 9A illustrates a square pattern of neighboringsamples for a block 900 in accordance with the present disclosure. Thesquare pattern is a combination pattern and includes, for a currentpixel 909, both the cross (902, 904, 906, 908) and diagonal patterns(901, 903, 905, 907) together. In one embodiment, other additionalfilter patterns may also be used. FIG. 9B illustrates a vertical-cross920, a horizontal-cross 940, a diagonal-left 960 and a diagonal-right980 patterns used in a filter in accordance with an embodiment of thepresent disclosure. The vertical-cross pattern 920 has the shape of anasymmetric cross for which the branch in the vertical direction or axisis longer. The horizontal-cross pattern 940 has the shape of anasymmetric cross for which the branch in the horizontal direction oraxis is longer. The diagonal-left pattern 960 has the shape of anasymmetric “X”, which is longer along the left leaning (or left)diagonal. The diagonal-right pattern 980 has the shape of an asymmetric“X”, which is longer along the right leaning (or right) diagonal. In oneembodiment, additional patterns which are a combination of any of thepatterns in FIGS. 2, 7, 9A and 9B may be used. For example, acombination of the vertical-cross 920 and the diagonal 700 may be used.

In the JVET-F0034 document, pixels at the perimeter of the block includepartial cross patterns in order to avoid using/processing samplesoutside the block, which may add complexity (e.g., memory, bandwidthaccess). Similarly, according to the present disclosure, perimeterpixels may utilize partial patterns, e.g., partial cross, partialdiagonal and/or partial square patterns. In one embodiment, partialsquare patterns may be utilized in perimeter pixels of blocks for whichinterior pixels use a cross pattern and/or diagonal pattern. In oneembodiment, partial square patterns are used for all perimeter pixels.In one embodiment, perimeter pixels process pixels from adjacent blocksresulting in added complexity.

FIG. 10 illustrates some exemplary partial patterns of neighboringsamples used in a filter in accordance with an embodiment of the presentdisclosure. In block 1000, the partial patterns are identified by therespective center perimeter pixel 1010-1080. Partial patterns 1010 and1040 are partial square patterns; 1020, a partial diagonal pattern;1030, a partial diagonal-right pattern; 1050 and 1080 are a partialcombination of a diagonal and a vertical cross pattern; 1060 is apartial horizontal cross pattern and 1070 is a partial cross pattern. Inone embodiment, additional patterns which are partial combinations ofany of the patterns in FIGS. 2, 7, 9A and 9B may be used. It may bedesirable to use as many partial combination patterns as possible (e.g.,partial square pattern) in block 1000, in order to include more pixelsin the filtering of perimeter pixels.

The filter pattern shape may vary per sequence, per picture, per sliceor per block. Advantageously, the filter pattern shape may vary insidethe block, per sample or per group of samples (e.g., 4×4 sub-block). Theselection may be made at the encoder, for instance, based on thecomputation of the local reconstructed signal properties (such as thelocal gradients).

The local gradient parameters may be calculated as follows:g _(h)=Σ_((x,y) in D) _(h) |2*I(x,y)−I(x−1,y)−I(x+1,y)|  (4)g _(v)=Σ_((x,y) in D) _(v) |2*I(x,y)−I(x,y−1)−I(x,y+1)|  (5)g _(d1)=Σ_((x,y) in D) _(d1) |2*I(x,y)−I(x−1,y−1)−I(x+1,y+1)|  (6)g _(d2)=Σ_((x,y) in D) _(d2) |2*I(x,y)−I(x−1,y+1)−I(x+1,y−1)|  (7)where D_(x)(x)|_(x=h,y,d1,d2) is the domain on which the gradient g_(x)is computed (e.g., the block).

The property derived from these parameters used for the selection mayfor instance be based on the following process:i. If (g _(h) +g _(v))<λ*(g _(d1) +g _(d2)) then the cross pattern isused;  (8)

ii. otherwise the diagonal pattern is used.

where λ is a given parameter. In one embodiment, λ may be, e.g., equalto 1.

In one embodiment, the property may be another function of the localgradients, e.g., a nonlinear function.

In one embodiment, λ may depend on the block shape and/or dimensions,for instance being lower for rectangular blocks (e.g. 8×2, 16×4) andlarger for square blocks.

In one embodiment, λ may depend on decisions made for neighboringblocks. For example, if the neighboring blocks are coded with intravertical direction, then the cross pattern is more likely to be chosenand λ may be increased. On another example, if the diagonal pattern isused for the neighboring blocks, then the diagonal pattern is morelikely to be chosen and λ may be decreased.

In one embodiment, the selection of the filter pattern may further be afunction of the block shape and/or dimensions. For example, somepatterns may be enabled or forbidden depending on the block shape.Blocks may be classified according to their shape and/or dimensions, andfor each shape/dimension, a predefined set of patterns is possible, asillustrated below:

Block class 1→set 1={pattern1, pattern2, pattern3, pattern4}

Block class 2→set 2={pattern1, pattern3, pattern5}

Block class 3→set 3={pattern3, pattern4}

For example, pattern 1 may be the cross pattern, pattern 2 may be thesquare pattern, pattern 3 may be the diagonal pattern, pattern 4 may bethe partial square pattern and pattern 5 may be the partial crosspattern. For example, for square (4×4, 8×8, 16×16 . . . ) and thickrectangles (8×4, 4×8, 8×16, 16×8 . . . ), cross and diagonal patternsmay be enabled, while for thin rectangular shapes/dimensions such as8×2, 2×8, 16×4, 4×16, 4×32, 32×4, only the cross pattern may be enabled.In another example, considering the pattern shapes of FIG. 9b , forrectangular vertical large blocks (e.g., 4×16, 4×32, 8×32) the verticalcross shape may be enabled, for square and rectangular small blocks(e.g., 4×4, 8×8) cross and diagonal shapes may be enabled, forrectangular horizontal large blocks (e.g., 16×4, 32×4, 32×8) thehorizontal cross shape may be enabled, for square and rectangular largeblocks (e.g., 16×16, 32×32, 8×16, 16×8, 16×32, 32×16) vertical-cross,horizontal-cross, diagonal-left and diagonal-right may be enabled.

In one embodiment of the present disclosure, other types of filters withsimilar complexity may be utilized. In one embodiment, other types ofnonlinear filters with similar complexity may be utilized. For example,a filter may have the same number of weights and use the same filterpatterns as the BLF, but the weights may not necessarily satisfyequations 1 to 3 and may assume general values. In one embodiment, theweights may have equal values.

In one embodiment, the weights may be power normalized such that the sumof their squares may be equal to 1.

In one embodiment, the filter pattern shape to use for filtering may becoded and/or included and/or signaled/identified in the bitstream as asyntax element or a group of syntax elements. The filter pattern may becoded for each picture (PPS), each slice (slice header) or for eachblock or group of blocks or CU depth (e.g., 64×64).

For instance, a syntax element called pattern_index may be coded per CU.When using two patterns, a flag may be used to differentiate thepatterns.

In one embodiment, when more than 2 patterns exist and the current blockhas more than 2 neighbors, a merging process may be used to save codingbits. An example of syntax is given in Table 1, where the merging with aneighboring block may be utilized, that is, the pattern from a neighborblock may be reused for a current block. This case may be advantageouswhen the number of neighbors (M bit representation or up to 2^(M)) isinferior to the number of patterns (N bit representation, or up to2^(N)), that is, M<N.

TABLE 1 num. bits merge_flag u1 If (merge_flag) { neighbor_index uMpattern_to_use = neighbor_pattern_index } else { pattern_index uNpattern_to_use = pattern_index }

The syntax in Table 1 includes the merge_flag (1 bit) and either theneighbor_index (M bits) or the pattern_index (N bits). If merge_flag hasa first value (e.g., the value of ‘1’), then merge_flag andneighbor_index are explicitly encoded (included and encoded) in theencoded bitstream. If merge_flag has a second value (e.g., the value of‘0’), then merge_flag and pattern index are explicitly encoded in thebitstream. In one embodiment, at least one of the syntax elements arejust included in the encoded bitstream but not encoded.

In one embodiment, other values of the merge_flag may be utilized, e.g.,inverse values of merge_flag from the values used in Table 1. In oneembodiment, merge_flag, neighbor_index and pattern index are included inthe encoded bitstream. In one embodiment, merge_flag, neighbor_index andpattern index are explicitly encoded in the encoded bitstream. In oneembodiment, at least one of the syntax elements are just included in theencoded bitstream but not encoded.

The ‘neighbor_index’ represents the position of one neighbor in thelist. Typically, a list of neighbors is built. For example,neighbor_index=0 may represent a left neighbor block; neighbor_index=1may represent above neighbor block; neighbor_index=2 may representtop-left neighbor block and neighbor_index=3 may represent top-rightneighbor block. If one neighbor has the same pattern as another neighborin the list, then it is removed from the list. Current block uses thesame pattern (neighbor_pattern_index) as the one used by neighbor_indexif merge_flag is equal to ‘1’. If merge_flag is equal to ‘0’, then thepattern to use is pattern_index.

In one embodiment, when more than two patterns exist and the currentblock has only two neighbors (left and top), the syntax may be as shownin Table 2.

The syntax in Table 2 includes the merge_flag (1 bit) and either theleft_flag (1 bit) or the pattern_index (N bits). If merge_flag has afirst merge value (e.g., the value of ‘1’), then merge_flag andleft_flag are explicitly encoded in the encoded bitstream. If merge_flaghas a second merge value (e.g., the value of ‘0’), then merge flag andthe pattern index are explicitly encoded in the bitstream. In oneembodiment, at least one of the syntax elements are just included in theencoded bitstream but not encoded.

In addition, if the left_flag has a first left value (e.g., the value of‘1’), then the left_pattern_index is to be chosen, that is, the patternof the neighbor to the left of the block. If the left_flag has a secondleft value (e.g., the value of ‘0’), then the pattern of the neighbor onthe top of the block is to be chosen.

In one embodiment, other values of merge_flag may be utilized, e.g.,inverse values of merge_flag from the values used in Table 2. In oneembodiment, other values of left_flag may be utilized, e.g., inversevalues of left_flag from the values used in Table 2. In one embodiment,merge_flag, left_flag and pattern index are explicitly encoded in theencoded bitstream. In one embodiment, at least one of the syntaxelements are just included in the encoded bitstream but not encoded.

Whether a syntax table according to Table 1 or Table 2 is included inthe encoded bitstream for a block, at the decoder, the syntax table isrecovered in order to identify the appropriate filter pattern for theblock.

TABLE 2 num. bits merge_flag u1 If (merge_flag) { left_flag u1pattern_to_use = left_flag ? left_pattern_index : top_pattern_index }else { pattern_index uN pattern_to_use = pattern_index }

FIG. 11 illustrates a flowchart 1100 of an exemplary method of videoencoding in accordance with one embodiment of the present disclosure.The method 1100 includes, at step 1110, accessing a reconstructed blockcorresponding to a block in a picture of a video. The picture may bealternately called first picture. Then, at step 1120, the method 1100includes determining at least one filter pattern based on a property ofthe block. Moreover, at step 1130, the method 1100 includes filteringthe reconstructed block according to the at least one filter pattern.Steps 1110 to 1130 may be performed, e.g., by encoder 600. Inparticular, steps 1120 and 1130 may be performed by, e.g., block filter685.

According to one embodiment, at step 1140, the method 1100 may furtherinclude encoding a second block of the video based on the filteredblock. The second block may be a second block of the picture (in thecase of intra prediction) or a second block of a second picture of thevideo (in the case of intra prediction). Step 1140 may be performed,e.g., by encoder 600. In particular, step 1140 may be performed by atleast one of intra-prediction module 660 and inter-prediction relatedmodules 670, 675 and 680. Step 1140 of encoding may be optional,bypassed or removed, since it may be performed by another device.

According to one embodiment of the method, the property of the block mayinclude a function of at least one local gradient for the reconstructedblock. The local gradients may be determined according to, e.g.,equations 4-7. The function may satisfy equation 8 (items i, ii).

According to one embodiment of the method, the property of the block mayinclude a function of at least one local gradient for the reconstructedblock at the video encoder. The local gradients may be determinedaccording to, e.g., equations 4-7. The function may satisfy equation 8(items i, ii).

According to one embodiment of the method, at least one syntax elementindicating the filter pattern may be included in the encoded video. Thesyntax element(s) may be encoded or not encoded. The syntax element(s)may satisfy, e.g., Table 1 or Table 2.

According to one embodiment of the method, the at least one syntaxelement may be shared among one of a plurality of pixels in the block, aplurality of blocks in the picture, a plurality of slices in the pictureand a plurality of pictures in the video. The syntax element(s) maysatisfy, e.g., Table 1 or Table 2.

According to one embodiment of the method, the at least one filterpattern may be determined at the video decoder by retrieving the atleast one syntax element from the encoded video. The syntax element(s)may satisfy, e.g., Table 1 or Table 2.

According to one embodiment of the method, the property of the block mayfurther include a shape of the block.

According to one embodiment of the method, the property of the block mayfurther include a function of a shape of the block.

According to one embodiment of the method, the filtering may beperformed by a nonlinear filter. For example, the filter may be abilateral filter of equations 1-3, or a modified version of thebilateral filter where the weights are based on a Gaussian distribution.

According to one embodiment of the method, the filtering may beperformed by a bilateral filter. The bilateral filter may satisfyequations 1-3.

According to one embodiment of the method, the at least one filterpattern may be at least one of a cross pattern, a diagonal pattern, asquare pattern, a partial cross pattern, a partial diagonal pattern anda partial square pattern.

According to one embodiment of the method, the at least one filterpattern may be at least one of a cross pattern, a diagonal pattern, asquare pattern, a horizontal-cross pattern, a vertical-cross pattern, adiagonal-left pattern, a diagonal-right pattern, a partial crosspattern, a partial diagonal pattern, a partial square pattern, a partialhorizontal-cross pattern, a partial vertical-cross pattern, a partialdiagonal-left pattern and a partial diagonal-right pattern.

According to one embodiment of the method, the at least one filterpattern may be at least one of a cross pattern, a diagonal pattern and asquare pattern when a pixel is an interior pixel of the block.

According to one embodiment of the method, the at least one filterpattern may be at least one of a partial cross pattern, a partialdiagonal pattern and a partial square pattern when a pixel is aperimeter pixel of the block.

According to one embodiment of the method, the at least one filterpattern may be at least one of a cross pattern, a diagonal pattern, asquare pattern, a horizontal-cross pattern, a vertical-cross pattern, adiagonal-left pattern and a diagonal-right pattern when a pixel is aninterior pixel of the block.

According to one embodiment of the method, the at least one filterpattern may be at least one of a partial cross pattern, a partialdiagonal pattern, a partial square pattern, a partial horizontal-crosspattern, a partial vertical-cross pattern, a partial diagonal-leftpattern and a partial diagonal-right pattern when a pixel is a perimeterpixel of the block.

According to one embodiment of the method, the block may bereconstructed by accessing a transformed and quantized predictionresidual associated with the block, reconstructing the residual byinverse quantizing and inverse transforming the residual and adding thereconstructed residual to a prediction for the block. The inversequantizing, inverse transforming and reconstructing may be performed by,e.g., modules 640, 650 and 655 of encoder 600, respectively.

According to one embodiment of the method, the filtered block may beprovided to a at least one of an intra prediction (e.g., module 660) andinter prediction (e.g., modules 665, 680 and 675) module.

According to one embodiment of the method, the filtered block may beprovided to a reference picture buffer (e.g., module 680).

According to one embodiment of the method, the filtered block may beprovided to an in-loop filter module (e.g., module 665).

According to one embodiment of the method, the filtered block may beprovided as a decoded block at the encoder output.

According to one embodiment of the method, the filtered block may beprovided to a reference picture buffer (e.g., module 680).

According to one embodiment, the method may further include receivingthe picture, partitioning the picture into a plurality of blocksincluding the block, determining a prediction residual for the block,transforming and quantizing the residual to obtain a plurality oftransform coefficients and entropy encoding the residual. The steps oftransforming and quantizing may be performed by, e.g., modules 625 and630 of encoder 600. The step of entropy encoding may be performed by,e.g., module 645 of encoder 600. The steps of receiving, transformingand quantizing may be optional, bypassed or removed, since they may havebeen previously performed by another device and/or the results may havebeen stored in memory.

It is to be understood that any of the embodiments of the method 1100described above may be implemented by encoder 600. The blocks of encoder600 may be implemented by hardware (e.g., integrated circuits) or insoftware, stored in memory and executed by a processor.

Decoding

FIG. 12 illustrates a simplified block diagram of an exemplary videodecoder 1200 in accordance with an embodiment of the present disclosure.The video decoder 1200 may be included in a receiver in a communicationsystem. Video decoder 1200 generally performs a decoding pass reciprocalto the encoding pass performed by the video encoder 600 as described inFIG. 6. In particular, the input of the decoder 1200 includes a videobitstream, which may be generated by the video encoder 600. Thebitstream is first entropy decoded (module 1230) to obtain transformcoefficients, motion vectors, syntax elements and other codedinformation. The transform coefficients are de-quantized (module 1240)and inverse transformed (module 1250) to decode residuals. The decodedresiduals are then combined (module 1255) with a predicted sample block(also known as a predictor) to obtain a decoded/reconstructed imageblock. The predicted sample block may be obtained (module 1270) fromintra prediction (module 1260) or motion-compensated prediction (i.e.,inter prediction) (module 1275). A block filter 1285 may filter thereconstructed block (e.g., using a bilateral filter) after combiner(also called reconstruction module) 1265. An in-loop filter (i.e., afilter within the prediction loop, module 1265) may be applied to theblock filtered reconstructed slice or image/picture/frame. The in-loopfilter may include a deblocking filter and a SAO filter. In in-loopfiltering, the filtering process may be performed, e.g., after an entireslice or image/picture/frame has been reconstructed, all-in-one, so thatthe filtered samples can be used for Inter-prediction. Hence,post-filtering 150 may be applied to in-filter(s) 1265. Notice, however,that block filtering 100 may also be utilized in in-loop filter(s) 1265.The filtered image is stored in a reference picture buffer 1280.

The modules of video decoder 1200 may be implemented in software andexecuted by a processor, or may be implemented by well-known circuits byone skilled in the art of compression. In particular, video encoder 1200may be implemented as an integrated circuit (IC), alone or combined withvideo decoder 600 as a codec.

The modules of video decoder 1200 are also present in other videodecoders (e.g., HEVC decoders), except for the differences described inthe present disclosure, particularly, differences in the block sizes andshapes, and differences in the presence of block filter module 685, aswill be described in greater detail in the following paragraphs.

It is to be understood that, in additional (not shown) embodimentsaccording to the present disclosure, block filter 685 may be placed inone of: inside the intra prediction module 660, inside in-loop filter(s)module 665, inside both the intra prediction module 660 and the in-loopfilter(s) module 665, or inside the combiner module 655.

Since the decoder 1200 of the present disclosure also includes blockfiltering, all the embodiments of block filtering described associatedwith encoder 600 also apply for decoder 1200. In addition, when thefilter pattern(s) is(are) encoded and/or included and/orsignaled/identified in the bitstream as syntax element(s), the decodermay retrieve the syntax element(s) from the encoded bitstream in orderto obtain the filter pattern prior to performing the block filtering,without the need for computation or selection.

In one embodiment, the computation associated with the filter patternchoice is also made at the decoder so that (the index of) the pattern touse is not coded but derived from reconstructed samples. In that case,the samples to use may not go outside the block to reduce bandwidthmemory access. For instance, if gradients are computed at the decoder,only the pixels located inside the block are considered.

FIG. 13 illustrates a flowchart 1300 of an exemplary method of videodecoding in accordance with one embodiment of the present disclosure.The method 1300 includes, at step 1310, accessing a reconstructed blockcorresponding to a block in a picture of an encoded video. The picturemay be alternately called first picture. Then, at step 1320, the method1300 includes determining at least one filter pattern based on aproperty of the block. Moreover, at step 1330, the method 1300 includesfiltering the reconstructed block according to the at least one filterpattern. Steps 1310 to 1330 may be performed, e.g., by decoder 1200. Inparticular, steps 1320 and 1330 may be performed by, e.g., block filtermodule 1285.

According to one embodiment, at step 1340, the method 1300 may furtherinclude decoding a second block of the encoded video based on thefiltered block. The second block may be a second block of the picture(in the case of intra prediction) or a second block of a second pictureof the encoded video (in the case of inter prediction). Step 1140 may beperformed, e.g., by encoder 600. In particular, step 1140 may beperformed by at least one of intra-prediction module 1260 andinter-prediction related modules 1270, 1275 and 1280. Step 1140 ofencoding may be optional, bypassed or removed, since it may be performedby another device.

According to one embodiment of the method, the property of the block mayinclude a function of at least one local gradient for the reconstructedblock at the video decoder, e.g., decoder 1200. The local gradients maybe determined according to, e.g., equations 4-7. The function maysatisfy equation 8 (items i, ii).

According to one embodiment of the method, the property of the block mayinclude a function of at least one local gradient for the reconstructedblock at a corresponding video encoder (e.g., encoder 600). The localgradients may be determined according to, e.g., equations 4-7. Thefunction may satisfy equation 8 (items i, ii).

According to one embodiment of the method, at least one syntax elementindicating the filter pattern may be included in the encoded video. Thesyntax element(s) may be encoded or not encoded. The syntax element(s)may satisfy, e.g., Table 1 or Table 2.

According to one embodiment of the method, the at least one syntaxelement may be shared among one of a plurality of pixels in the block, aplurality of blocks in the picture, a plurality of slices in the pictureand a plurality of pictures in the video. The syntax element(s) maysatisfy, e.g., Table 1 or Table 2.

According to one embodiment of the method, the at least one filterpattern may be determined at the video decoder by retrieving the atleast one syntax element from the encoded video. The syntax element(s)may satisfy, e.g., Table 1 or Table 2.

According to one embodiment of the method, the property of the block mayfurther include a shape of the block.

According to one embodiment of the method, the property of the block mayfurther include a function of a shape of the block.

According to one embodiment of the method, the filtering may beperformed by a nonlinear filter. For example, the filter may be abilateral filter of equations 1-3, or a modified version of thebilateral filter where the weights are based on a Gaussian distribution.

According to one embodiment of the method, the filtering may beperformed by a bilateral filter. The bilateral filter may satisfyequations 1-3.

According to one embodiment of the method, the at least one filterpattern may be at least one of a cross pattern, a diagonal pattern, asquare pattern, a partial cross pattern, a partial diagonal pattern anda partial square pattern.

According to one embodiment of the method, the at least one filterpattern may be at least one of a cross pattern, a diagonal pattern, asquare pattern, a horizontal-cross pattern, a vertical-cross pattern, adiagonal-left pattern, a diagonal-right pattern, a partial crosspattern, a partial diagonal pattern, a partial square pattern, a partialhorizontal-cross pattern, a partial vertical-cross pattern, a partialdiagonal-left pattern and a partial diagonal-right pattern.

According to one embodiment of the method, the at least one filterpattern may be at least one of a cross pattern, a diagonal pattern and asquare pattern when a pixel is an interior pixel of the block.

According to one embodiment of the method, the at least one filterpattern may be at least one of a partial cross pattern, a partialdiagonal pattern and a partial square pattern when a pixel is aperimeter pixel of the block.

According to one embodiment of the method, the at least one filterpattern may be at least one of a cross pattern, a diagonal pattern, asquare pattern, a horizontal-cross pattern, a vertical-cross pattern, adiagonal-left pattern and a diagonal-right pattern when a pixel is aninterior pixel of the block.

According to one embodiment of the method, the at least one filterpattern may be at least one of a partial cross pattern, a partialdiagonal pattern, a partial square pattern, a partial horizontal-crosspattern, a partial vertical-cross pattern, a partial diagonal-leftpattern and a partial diagonal-right pattern when a pixel is a perimeterpixel of the block.

According to one embodiment of the method, the block may bereconstructed by accessing a transformed and quantized predictionresidual associated with the block, reconstructing the residual byinverse quantizing and inverse transforming the residual and adding thereconstructed residual to a prediction for the block. The inversequantizing, inverse transforming and reconstructing may be performed by,e.g., modules 1240, 1250 and 1255, respectively.

According to one embodiment of the method, the filtered block may beprovided to a at least one of an intra prediction (e.g., module 1260)and inter prediction (e.g., modules 1265, 1280 and 1275) module.

According to one embodiment of the method, the filtered block may beprovided to a reference picture buffer (e.g., module 1280).

According to one embodiment of the method, the filtered block may beprovided to an in-loop filter module (e.g., module 1265).

According to one embodiment of the method, the filtered block may beprovided as a decoded block at the decoder output.

According to one embodiment, the method may further include receivingthe encoded picture, entropy decoding the block, inverse transformingthe block to obtain decoded residuals, combining the decoded residualswith a predicted sample block to obtain a decoded/reconstructed imageblock. The transform coefficients may be further inverse quantized priorto inverse transformed. The steps of entropy decoding, inversetransforming and inverse quantizing may be performed by, e.g., modules1230, 1250 and 1240 of decoder 1200, respectively. The steps ofreceiving, entropy decoding, inverse transforming and inversequantizing, and combining may be optional, bypassed or removed, sincethey may have been previously performed by another device and/orprovided to another device, or the results may have been retrieved fromand/or stored in memory.

It is to be understood that any of the embodiments of the method 1300described above may be implemented by decoder 1200. The modules ofdecoder 1200 may be implemented by hardware (e.g., integrated circuits)or in software, stored in memory and executed by a processor.

FIG. 14 illustrates a block diagram 1400 of an exemplary system in whichvarious aspects of the exemplary embodiments of the present disclosuremay be implemented. System 1400 may be embodied as a device includingthe various components described below and is configured to perform theprocesses described above. Examples of such devices, include, but arenot limited to, personal computers, laptop computers, smartphones, smartwatches, tablet computers, digital multimedia set top boxes, digitaltelevision receivers, personal video recording systems, connected homeappliances, and servers. System 1400 may be communicatively coupled toother similar systems, and to a display via a communication channel asshown in FIG. 14 and as known by those skilled in the art to implementthe exemplary video system described above. System 1400 may implementencoder 600, decoder 1200 or both, independently or jointly. Moreover,system 1400 may implement and be configured to execute any of theprocesses of the present disclosure, including method 1100 and/or 1300,independently or jointly.

The system 1400 may include at least one processor 1410 configured toexecute instructions loaded therein for implementing the variousprocesses as discussed above. Processor 1410 may include embeddedmemory, input output interface and various other circuitries as known inthe art. The system 1400 may also include at least one memory 1420(e.g., a volatile memory device such as RAM, a non-volatile memorydevice such as ROM). System 1400 may additionally include a storagedevice 1440, which may include non-volatile memory, including, but notlimited to, an erasable programmable read-only memory (EPROM), ROM, aprogrammable read-only memory (PROM), a dynamic RAM (DRAM), a static RAM(SRAM), flash memory, magnetic disk drive, and/or optical disk drive.The storage device 1440 may comprise an internal storage device, anattached storage device and/or a network accessible storage device, asnon-limiting examples. System 1400 may also include an encoder/decodermodule 1430 configured to process data to provide an encoded video ordecoded video.

Encoder/decoder module 1430 represents the module(s) that may beincluded in a device to perform the encoding and/or decoding functions,for example, according to FIGS. 6 and 12, respectively. As is known inthe art of compression, a device may include one or both of the encodingand decoding modules. Additionally, encoder/decoder module 1430 may beimplemented as a separate element of system 1400 or may be incorporatedwithin processors 1410 as a combination of hardware and software asknown to those skilled in the art. For example, encoder/decoder module1430 may be implemented as one or two separate integrated circuitsand/or field-programmable gate array (FPGA).

Program code to be loaded onto processors 1410 to perform the variousprocesses described hereinabove may be stored in storage device 1440 andsubsequently loaded onto memory 1420 for execution by processors 1410.In accordance with the exemplary embodiments of the present disclosure,one or more of the processor(s) 1410, memory 1420, storage device 1440and encoder/decoder module 1430 may store one or more of the variousitems during the performance of the processes discussed herein above,including, but not limited to the input video, the decode video, thebitstream, equations, formula, matrices, variables, operations, andoperational logic.

The system 1400 may also include communication interface 1450 thatenables communication with other devices via communication channel 1460.The communication interface 1450 may include, but is not limited to atransceiver configured to transmit and receive data from communicationchannel 1460. The communication interface may include, but is notlimited to, a modem or network card and the communication channel may beimplemented within a wired and/or wireless medium. The variouscomponents of system 1400 may be connected or communicatively coupledtogether using various suitable connections, including, but not limitedto internal buses, wires, and printed circuit boards.

The exemplary embodiments according to the present disclosure may becarried out by computer software executed by the processor 1410 or byhardware, or by a combination of hardware and software. As anon-limiting example, the exemplary embodiments according to the presentdisclosure may be implemented by one or more integrated circuits. Thememory 1420 may be of any type appropriate to the technical environmentand may be implemented using any appropriate data storage technology,such as optical memory devices, magnetic memory devices,semiconductor-based memory devices, fixed memory and removable memory,as non-limiting examples. The processor 1410 may be of any typeappropriate to the technical environment, and may encompass one or moreof microprocessors, general purpose computers, special purpose computersand processors based on a multi-core architecture, as non-limitingexamples.

The implementations described herein may be implemented in, for example,a method or a process, an apparatus, a software program, a data stream,or a signal. Even if only discussed in the context of a single form ofimplementation (for example, discussed only as a method), theimplementation of features discussed may also be implemented in otherforms (for example, an apparatus or program). An apparatus may beimplemented in, for example, appropriate hardware, software, andfirmware. The methods may be implemented in, for example, an apparatussuch as, for example, a processor, which refers to processing devices ingeneral, including, for example, a computer, a microprocessor, anintegrated circuit, or a programmable logic device. Processors alsoinclude communication devices, such as, for example, computers, cellphones, portable/personal digital assistants (PDAs), and other devicesthat facilitate communication of information between end-users.

According to an aspect of the present disclosure, an apparatus 1400 forvideo encoding is provided, the apparatus including a processor 1410,and at least one memory 1420, 1440 coupled to the processor, theprocessor 1410 being configured to perform any of the embodiments of themethod of video encoding 1100 described above.

According to an aspect of the present disclosure, an apparatus 1400 forvideo decoding is provided, the apparatus including a processor 1410,and at least one memory 1420, 1440 coupled to the processor, theprocessor 1410 being configured to perform any of the embodiments of themethod of video decoding 1300 described above.

According to an aspect of the present disclosure, an apparatus for videoencoding is provided including means for accessing a reconstructed blockcorresponding to a block in a picture of a video, means for determiningat least one filter pattern based on a property of the block and meansfor filtering the reconstructed block according to the at least onefilter pattern. The apparatus may further include means for encoding asecond picture of the video based on the filtered block. The videoencoders of FIGS. 6 and 14 may include the structure or means of theapparatus, particularly, blocks 660, 665, 655, 670, 675, 680, 1410 and1430. The apparatus for video encoding may perform any of theembodiments of the method 1100 of video encoding.

According to an aspect of the present disclosure, an apparatus for videodecoding is provided including means for accessing a reconstructed blockcorresponding to a block in a picture of an encoded video, means fordetermining at least one filter pattern based on a property of the blockand means for filtering the reconstructed block according to the atleast one filter pattern The apparatus may further include means fordecoding a second picture of the encoded video based on the filteredblock. FIGS. 12 and 14 may include the structure or means of theapparatus for video decoding, particularly, blocks 1260, 1265, 1255,1275, 1280, 1410 and 1430. The apparatus for video decoding may performany of the embodiments of the method 1300 of video encoding.

As will be evident to one of skill in the art, implementations mayproduce a variety of signals formatted to carry information that may be,for example, stored or transmitted. The information may include, forexample, instructions for performing a method, or data produced by oneof the described implementations. For example, a signal may be formattedto carry the bitstream of a described embodiment. Such a signal may beformatted, for example, as an electromagnetic wave (for example, using aradio frequency portion of spectrum) or as a baseband signal. Theformatting may include, for example, encoding a data stream andmodulating a carrier with the encoded data stream. The information thatthe signal carries may be, for example, analog or digital information.The signal may be transmitted over a variety of different wired orwireless links, as is known. The signal may be stored on aprocessor-readable medium.

According to an aspect of the present disclosure, a signal including abitstream formatted to include encoded data representative of a block ofa picture, the encoded data encoded according to any of the embodimentsof the method 1100 of video encoding.

According to an aspect of the present disclosure, a bitstream formattedto include encoded data representative of a block of a picture, theencoded data encoded according to any of the embodiments of the method1100 of video encoding.

Moreover, any of the methods 1100 and/or 1300 may be implemented as acomputer program product (independently or jointly) comprising computerexecutable instructions which may be executed by a processor. Thecomputer program product having the computer-executable instructions maybe stored in the respective transitory or non-transitorycomputer-readable storage media of the system 1400, encoder 600 and/ordecoder 1200.

According to an aspect of the present disclosure, a computer programproduct is provided including program code instructions for performingany of the embodiments of any of the methods 1100 and/or 1300(independently or jointly) of the present disclosure.

It is important to note that one or more of the elements in theprocesses 1100 and/or 1300 may be combined, performed in a differentorder, or excluded in some embodiments while still implementing theaspects of the present disclosure. Other steps may be performed inparallel, where the processor does not wait for a full completion of astep before starting another.

Furthermore, aspects of the present disclosure can take the form of acomputer-readable storage medium. Any combination of one or morecomputer-readable storage medium(s) may be utilized. A computer-readablestorage medium can take the form of a computer-readable program productembodied in one or more computer-readable medium(s) and havingcomputer-readable program code embodied thereon that is executable by acomputer. A computer-readable storage medium as used herein isconsidered a non-transitory storage medium given the inherent capabilityto store the information therein as well as the inherent capability toprovide retrieval of the information therefrom. A computer-readablestorage medium may be, for example, but is not limited to, anelectronic, magnetic, optical, electromagnetic, infrared, orsemiconductor system, apparatus, or device, or any suitable combinationof the foregoing.

It is to be appreciated that the following list, while providing morespecific examples of computer-readable storage mediums to which thepresent disclosure may be applied, is merely an illustrative and notexhaustive listing as is readily appreciated by one of ordinary skill inthe art. The list of examples includes a portable computer diskette, ahard disk, a ROM, EPROM, Flash memory, a portable compact disc read-onlymemory (CD-ROM), an optical storage device, a magnetic storage device,or any suitable combination of the foregoing.

According to an aspect of the present disclosure, a computer-readablestorage medium carrying a software program is provided including programcode instructions for performing any of the embodiments of any of themethods of the present disclosure, including methods 1100 and/or 1300.

It is to be understood that reference to “one embodiment” or “anembodiment” or “one implementation” or “an implementation” of thepresent disclosure, as well as other variations thereof, mean that aparticular feature, structure, characteristic, and so forth described inconnection with the embodiment is included in at least one embodiment ofthe present disclosure. Thus, the appearances of the phrase “in oneembodiment” or “in an embodiment” or “in one implementation” or “in animplementation”, as well any other variations, appearing in variousplaces throughout the specification are not necessarily all referring tothe same embodiment.

Additionally, the present disclosure or its claims may refer to“determining” various pieces of information. Determining the informationmay include one or more of, for example, estimating the information,calculating the information, predicting the information, or retrievingthe information from memory.

Also, the present disclosure or its claims may refer to “providing”various pieces of information. Providing the information may include oneor more of, for example, outputting the information, storing theinformation, transmitting the information, sending the information,displaying the information, showing the information, or moving theinformation.

Moreover, the present disclosure or its claims may refer to “accessing”various pieces of information. Accessing the information may include oneor more of, for example, receiving the information, retrieving theinformation (for example, from memory), storing the information,processing the information, moving the information, copying theinformation, erasing the information, calculating the information,determining the information, predicting the information, or estimatingthe information.

Further, the present disclosure or its claims may refer to “receiving”various pieces of information. Receiving is, as with “accessing”,intended to be a broad term. Receiving the information may include oneor more of, for example, accessing the information, or retrieving theinformation (for example, from memory). Further, “receiving” istypically involved, in one way or another, during operations such as,for example, storing the information, processing the information,transmitting the information, moving the information, copying theinformation, erasing the information, calculating the information,determining the information, predicting the information, or estimatingthe information.

It is to be appreciated that the various features shown and describedare interchangeable. Unless otherwise indicated, a feature shown in oneembodiment may be incorporated into another embodiment. Further, thefeatures described in the various embodiments may be combined orseparated unless otherwise indicated as inseparable or not combinable.

As noted before, the functions of the various elements shown in thefigures may be provided through the use of dedicated hardware as well ashardware capable of executing software in association with appropriatesoftware. Also, when provided by a processor, the functions may beprovided by a single dedicated processor, by a single shared processor,or by a plurality of individual processors, some of which may be shared.

It is to be further understood that, because some of the constituentsystem components and methods depicted in the accompanying drawings arepreferably implemented in software, the actual connections between thesystem components or the process function blocks may differ dependingupon the manner in which the processes of present disclosure areprogrammed. Given the teachings herein, one of ordinary skill in thepertinent art will be able to contemplate these and similarimplementations or configurations of the present disclosure.

Although the illustrative embodiments have been described herein withreference to the accompanying drawings, it is to be understood that thepresent disclosure is not limited to those precise embodiments, and thatvarious changes and modifications may be effected therein by one ofordinary skill in the pertinent art without departing from the scope ofthe present disclosure. In addition, individual embodiments can becombined, without departing from the scope of the present disclosure.All such changes and modifications are intended to be included withinthe scope of the present disclosure as set forth in the appended claims.

The invention claimed is:
 1. A method of video encoding comprising:accessing a reconstructed block corresponding to a block in a picture ofa video; determining a plurality of filter patterns associated with ablock class to which said reconstructed block belongs, said block classbeing defined by at least one of a block shape and a block size;determining one filter pattern for at least one sample of saidreconstructed block among said plurality in function of a local gradientcomputed for said at least one sample, said filter pattern defining asupport of a bilateral filter; and filtering said at least one sampleusing said bilateral filter.
 2. The method according to claim 1, whereinin the case where a sum of horizontal and vertical gradients is below aweighted sum of diagonal gradients, said one filter pattern is a crosspattern and otherwise, said one filter pattern is a diagonal pattern. 3.The method according to claim 1, wherein at least one syntax elementindicating said one filter pattern is included in the encoded video,said at least one syntax element being shared among one of a pluralityof pixels in said block, a plurality of blocks in said picture, aplurality of slices in said picture and a plurality of pictures in saidvideo.
 4. The method according to claim 1, wherein said one filterpattern is at least one of a cross pattern, a diagonal pattern, a squarepattern, a horizontal-cross pattern, a vertical-cross pattern, adiagonal-left pattern, a diagonal-right pattern, a partial crosspattern, a partial diagonal pattern, a partial square pattern, a partialhorizontal-cross pattern, a partial vertical-cross pattern, a partialdiagonal-left pattern and a partial diagonal-right pattern.
 5. Anapparatus for video encoding comprising an electronic circuitry adaptedfor: accessing a reconstructed block corresponding to a block in apicture of a video; determining a plurality of filter patternsassociated with a block class to which said reconstructed block belongs,said block class being defined by at least one of a block shape and ablock size; determining one filter pattern for at least one sample ofsaid reconstructed block among said plurality in function of a localgradient computed for said at least one sample, said filter patterndefining a support of a bilateral filter; and filtering said at leastone sample using said bilateral filter.
 6. The apparatus according toclaim 5, wherein in the case where a sum of horizontal and verticalgradients is below a weighted sum of diagonal gradients, said one filterpattern is a cross pattern and otherwise, said one filter pattern is adiagonal pattern.
 7. The apparatus according to claim 5, wherein atleast one syntax element indicating said one filter pattern is includedin the encoded video, said at least one syntax element being sharedamong one of a plurality of pixels in said block, a plurality of blocksin said picture, a plurality of slices in said picture and a pluralityof pictures in said video.
 8. The apparatus according to claim 5,wherein said one filter pattern is at least one of a cross pattern, adiagonal pattern, a square pattern, a horizontal-cross pattern, avertical-cross pattern, a diagonal-left pattern, a diagonal-rightpattern, a partial cross pattern, a partial diagonal pattern, a partialsquare pattern, a partial horizontal-cross pattern, a partialvertical-cross pattern, a partial diagonal-left pattern and a partialdiagonal-right pattern.
 9. A method of video decoding comprising:accessing a reconstructed block corresponding to a block in a picture ofan encoded video; determining a plurality of filter patterns associatedwith a block class to which said reconstructed block belongs, said blockclass being defined by at least one of a block shape and a block size;determining one filter pattern for at least one sample of saidreconstructed block among said plurality in function of a local gradientcomputed for said at least one sample, said filter pattern defining asupport of a bilateral filter based on a property of said block; andfiltering said at least one sample using said bilateral filter.
 10. Themethod according to claim 9, wherein in the case where a sum ofhorizontal and vertical gradients is below a weighted sum of diagonalgradients, said one filter pattern is a cross pattern and otherwise,said one filter pattern is a diagonal pattern.
 11. The method accordingto claim 9, wherein said property of said block includes a function ofat least one local gradient for a corresponding reconstructed block ofsaid block.
 12. The method according to claim 9, wherein at least onesyntax element indicating said one filter pattern is included in theencoded video, said at least one syntax element being shared among oneof a plurality of pixels in said block, a plurality of blocks in saidpicture, a plurality of slices in said picture and a plurality ofpictures in said video.
 13. The method according to claim 9, whereinsaid one filter pattern is at least one of a cross pattern, a diagonalpattern, a square pattern, a horizontal-cross pattern, a vertical-crosspattern, a diagonal-left pattern, a diagonal-right pattern, a partialcross pattern, a partial diagonal pattern, a partial square pattern, apartial horizontal-cross pattern, a partial vertical-cross pattern, apartial diagonal-left pattern and a partial diagonal-right pattern. 14.An apparatus for video decoding comprising: wherein the apparatuscomprises electronic circuitry adapted for: accessing a reconstructedblock corresponding to a block in a picture of an encoded video;determining a plurality of filter patterns associated with a block classto which said reconstructed block belongs, said block class beingdefined by at least one of a block shape and a block size; determiningone filter pattern for at least one sample of said reconstructed blockamong said plurality in function of a local gradient computed for saidat least one sample, said filter pattern defining a support of abilateral filter based on a property of said block; and filtering saidat least one sample using said bilateral filter.
 15. The apparatusaccording to claim 14, wherein in the case where a sum of horizontal andvertical gradients is below a weighted sum of diagonal gradients, saidone filter pattern is a cross pattern and otherwise, said one filterpattern is a diagonal pattern.
 16. The apparatus according to claim 14,wherein said property of said block includes a function of at least onelocal gradient for a corresponding reconstructed block of said block.17. The apparatus according to claim 14, wherein at least one syntaxelement indicating said one filter pattern is included in the encodedvideo, said at least one syntax element being shared among one of aplurality of pixels in said block, a plurality of blocks in saidpicture, a plurality of slices in said picture and a plurality ofpictures in said video.
 18. The apparatus according to claim 14, whereinsaid one filter pattern is at least one of a cross pattern, a diagonalpattern, a square pattern, a horizontal-cross pattern, a vertical-crosspattern, a diagonal-left pattern, a diagonal-right pattern, a partialcross pattern, a partial diagonal pattern, a partial square pattern, apartial horizontal-cross pattern, a partial vertical-cross pattern, apartial diagonal-left pattern and a partial diagonal-right pattern.